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_ Ongoing Planning as Continuing | 
Program Develapment 


Ongoing planning is a cyclical process that keeps a program’ 


current with changing needs and conditions. A program using 
good ongoing planning (monitoring, evaluation, and decision 
making) is one that can be effectively managed, as adjust- 
ments in it are based on a Rnowledge of what is and what is 
not working in the program, and why. Decisions to change 
the program, therefore, can be made throughout the year as 
information is generated, rather than just once a year when 
plans are being written 


|. Monitoring the Implementation 
of the Program 


7 \ 
@ To what extent have provisions been made for monitoring the 
implementation of the program? 


© How clearly have thé following questions been answered? 
What information is important in judging how well the 
program is working? - 
© How and when can this information best be collected? 
Who should reew the information? 
© To what extent are the provisions for monitoring being carried 
- out? 


“@ How clearly established and timely are the interitn points for 
examining the program? 


doiig, and it cdn help you to illustrate the im- 
portance of monitoring to the whole planning 
process. 


You may recognize the above questions 
from the “Program Quality Review Instru- 
ment—1976/77" that recently was sent to 


California’$ schools. The questions themselves 
. indicate the general steps in the “monitoring” 
segment of an ongoing planning proctss. The 
cancept of monitoring certainly is pot new. 
The monitoring process can help you to 
organize some things your school already is 


. 


tay 


On the following pages are some Do’s and 
Don’ts that may help you recognize the 
monitoring that, you already are doing. The 
section after the Do’s and Don'ts is an outline 
of.a procedure to monitor the implementa- 
tion of Your program. 
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wt UV 7 Don's: 
\ 4 
@° DON’T look at all components and solu- 
tion procedures at once. You really can’t 
anyway, so don’t kid yourself. 


‘ 


@ DON’T assume that predetermined infor- . 


mation requirements are sufficient and ask | , 


only the questions that are mandated by 
the state or federal agencies. The most 
important information for you may not be 
the number of participants in each grade, 
number of personnel, test results for each 
component, or the extent to which objec- 
tives are achieved. it 


@ DON’T collect iriformation unless you have 
decided what you are*going to do with it. 


nek 7" ) 
ges Do’s”. —_ 


© DO choose to look at a new or controver- 
sial feature of the program; for example,: 
the solution procedures in reading or in 
staff development that may soon be con-— 
sidered by the school board, principal, 
school advisory committee, or staff for 
continuation, revision; or elimination. 


© DO specifically decide what you most need 
to’ know about how a certain feature of a 
program is working; for example, what 
evidence exists that: 


The strategy is doing what it is supposed 


to be doing. P 


“ The time, money, and personnel required 
* to do it are being used effectively. 
The strategy is better than’ previous or 
alternative program strategies. 
The strategy is having good effects on 
students. 


| 


@ DO ask any important questions that will 
aid you in making decisions to modify the 
program. You must decide what informa- 
tion you need. 


© DON’T collect ifformation that is different 
from that which Pou planned to collect. 
Neither should you try to collect HI the 
information available. 


@ DON’T collect information in a hapteazard 
way, such as: 


At various tintes; if the specific time the 
information is collected is important 

© From various types of people, if only 
teachers or parents or students are to 
respond 


With unclear or different directions for 
responses 
\ 


e 


DON’T organize and use@the information 
only at the end of the year. By the , a 
the year, if successful strategies have Been 
found, everyone should have been alerted 
so he or shgecan be adapting them, to his.or 


her part of the pregram. If strategies have, 


not worked out, theyeshould be modified 
early so that the resulting dam¥ge cin be 
minimized. 


DON'T assume that the distrct project 
directdy or the principal alone will be able 
to ensugB<{Nat all the information is col- 
lected. ~ ° ; 


DON’T expect’ that communication will be 
effective unless you continue to work on it. 
Communication is essential if decisions 


about the program are to,be made in an 
. a + ’ ° 

informed and enlightened way by those 
who, must make them. 


“~ . Don'ts . a 


at 


, Do’ 7 


@ DO select the same igformation that you 


decided you would c@llect, and gather it in 
the way that was planned. 


‘ 
, 


, 


DO collect informatign that will answer the - 
questions you have Chosen. For example: 
Are students reading better, quicker, and 
with more insight and comprehension? 


How much time does a particular strat- 
y egy require, and are people happier with _ 

it? 

What do students think about the strat- 

egy, and has their behavior changed? 

Are students showing more progress in 

mathenfatics at this date than they did 

last year at this same time? 


DO collect information early and_ fre- 
quently enough so\that decisions can be 
made throughout ‘the year as to whether an 
alternative strategy might be better. Choose 
specific months for evaluating the collected 
information that will help the decision- 
making process, such as November, Match, 
May, or whenever. 

i 


DO select a representative committée (or 
component committee) and a chairperson 
to be responsible for deciding how the 
information is to be collected and for 
ensuring that the information is collected. 


DO make communication one of this 
group’s high priorities so that ways can bt 
found to keep informed all those with a 
need to know what is going on. Keep 
people informed by méans of daily bulle- 
tins or presentations at weekly staff meet- 
ings. by rotating memberships on the com- .« 
mittee, more people can be kept informed. 


DO make sure that the collected informa- 
tion gets to the people who will evaluate it 
and that” the evaluation findings get to 
those who make the decisions. 


An Outline for Monitoring 


** Monitoring the implementation of the program amounts to observ- 
ing its progress systematically. 

Because all activities cannot be observed simultaneously, monitor- 
ing should allow for some Priorities to be set among the areas al the 
program that need to be examined. 

You may find it useful to adapt the following steps to your 
particular needs. You can.use them to monitor we implementation of 
‘your program. ‘ 


A. List priorities. A . 


L': 


dentify those issues that require information for future 
decisions pertaining to problems or areas of special interest. 
State the issues in question form so that decisions can be 
made about the information that is to be collected to help 
resolve the issues. 


B.' Consider timing and methods. 


C; 


D. 


Décide how and when the information can best be collected 
so ‘that those involved in ‘the collection can be‘trained and 
so that the actual collection does not conflict with other 
activities. 

Set dates for the’ analysis in advance. The dates should be 
appropnate for the time when. decisions will have to be 
mate to modify the program. 


Assign the responsibility for the monitoring. 


Select someone to be responsible for the monitoring.’ 

Use evaluation personnel to help design the monitoring. 
Establish a means of internal communication that will 
create a general awareness of ongoing planning. 


Use the procedures that have been established to collect the 
information. 


‘ @ 


Il. Evaluating Information 
‘ About the Program 


® How clearly have procedures for evaluation at interim points 


provided for: 


> Analyzing the collected information 
Interpreting the analysis 


™’ 
@ How thoroughly has the following information about the 
_ Program been evaluated: 


Qnality review of the school plan by state and/or district 


reviewers 


4. 


Py » 


Student assessment data and’ other measures of accomplish- 


iment from | the previous year 


' 


Information collected from monitoring the implementation 
‘ “ 


of the program 


y 
The preceding questions fr m the “Pro- 
gram Quality Review Instrument—1976/77”" 
refer to the part of the cyclical ongoing 
planning process that) is used/to evalfate the 
information collected about the program. 


Several cautions abdut evalfiation should be ° 


made at this time. They are /the following: (1) 
collection of information jis often followed 
directly by decision making while the organiz- 


ing, analyzing, and interpreting of the infor- 
mation is neglected; (2) often evaluation is_ 


‘carried out, but without regard to the sepa- 


rate steps of organizing and analyzing the 
information and interpreting the results; and 
(3) the cntique- of the plan, stydent assess- 


“ment data, and other pertinent information 


from the evaluation pr@eess may be over- 
looked. 

The following pages may clarify the evalda- 
tion portion of the ongoingyplanning process. 
They contain some Do's and Don'ts and an 
outline of the decision-making Steps. 
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@ DON’T evaluate collected information only 


once or twice a year. 


© DON’T organize and analyze the informa- 
tion without considering the questions you 
have asked about your program. 


@ DON’T organize test score data only on the 
basis of required reporting formats. 


@ DON’T organize information only in accor- 
dance with the usual quantitative break- 
outs, such as the following: 


Percentages of right and wrong answers © 


Percentages of each kind of response 


@ DON’T evaluate only the information that 
is collected threugh monitoring. 


© DO evaluate the information at times when 


results will be most useful for decision 
making. 


@ DO decide what criteria and levels of 


success or effectiveness are acceptable. 


@ DO organize the information so that the 


questions asked will be answered. Try to do 

the following: : 
Arrange téachers’ responses to an atti- 
tude questionnaire on the basis of clus- 
ters of related questions. 

© Group students’ scores in accordance 
with achievement, and look for differing 
patterns of right and wrong answers. 

> Group parents’ responses in accordance 
with some distifguishing chara¢teristic, 
such as upper-grade/lower-grade; and 
look for differing patterns of responses. 


® DO evaluate information obtained through 


the quality review of the school plan, the 
formal assessments of students, and the 
end-of-year data concerning the attainment 
of objectives and the effectiveness of solu- 
tion procedures. 


Don'ts 
@ DON’T explain the relationships between 
the categories by looking only at superficial 
reasons. 


@ DON’T be content with just reporting the 
scores as the findings. 


Do’s 


@ DO explain the relationships (or lack of 


them) between the selected categories, 
using various elements making up the con- 
text of the program, such as the following: 


School, staff, students, goals and objec- 
tives, budget, community, district, time’ 
of year, length of time in the reform 
effort, and use of volunteers or adults in 
the classroom. 

Specific type of program, such = as 
amount of student choice, multiage , 
grouping, cross-age tutors, degree of 
individualization in social studies, science, 
and the like. . 


New. staff, new building, earthquake 
safety, or late delivéry of materials. 


DO determine the impfications of the 
findings for future questions about the 
program: Implications should. be stated 
with specificity.” Examples could be the 
following: 


© Higher achieving groups score higher.on 
only one type of question. (Should more 
specific information on overachieving 
groups be collected?) 


© Only teachers in grade three are satisfted 


~with staff development activities. 
Parents most satisfied with individualiza- 
tion are those who have only one child 
in school. 


An Outline for Evaluation 

The evaluation of the ongoing planning process concerns the 
organizing, analyzing, and interpreting of the iaformation that has been 
collected through monitoring procedures. 

You may consider adapting the followirig steps to your particular 
needs. You can use them to ae the information collected,on your 
program. ae . + , Son 

A. Set up the evaluation procedures. 


1. Decide how to organize and analyze the collected tibet. 
tion so that the questions you have askéd about your . 
program will be answered by the results of your analysis. 

2. Decide what levels of tffectivepess and criteria fot success 
will be used. 


. Analyze the information that has been collected. . 


1. Analyze all of the ayailable ahd relevant information. Do 
not overlook plan critiques, end-of-year test results, or other 
measures of ¥apmplishment. 
Categorize the information, and make comparisoris between 
the categories you have chosen. Look at the kind and degree’, 
of relationships that exist between and within the nici 
C. Interpret the results of the analysis, \ 

1. Study the results to determine whether they are logical, 

* considering the questions you have asked about the pro- - 

gram. rd 

2. Determine the best explanation of the results with regard to 
‘ the context and the idiosyncrasies of the program. 

3. Decide if the results raise new questions about the imple- 
mentation of the program. \ 
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- IP Making Begicione. 
to Modify: the: Program 


- @ How hearty established are the procedures for making d decisions 
in order, to modify the program? , 
© To what extent has the decision-making" process iepearhid for: “ 
© Using evaluation results j ; 
Q., Determining need for change 
© Prioritizing those needs for change 


@ When need for change has been determined, how thorounly 
has the decision-making process provided for: ‘ 


> Developing alternative solution procedures - 


© Selecting from among alternatives 
° Implementing selected solution procedures 


© To what extent have the modifications which occurred this year 
come about a8 a result: of the established; decision-making 
| pemensherea? 


. « e . 
The preceding ‘questions from ‘the “Pro- change the program is usually the weakest 
gram Quality Review Instrument—1976/77" link in ‘planning. The following list of Do’s 
are those that relate to what ‘occurs after the and Don'ts may be used to review your 
evaluation *~ process has begun. The use of program’s progress. The outline which follows 
evaluation findings to |make the decision to. the list may be used as a reference. © . 
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Decision Making 


Don'ts 


DON’T forget that to make no decision is 
to make a decision. 


DON’T use a decision-making process that 
is unknown or misunderstood by those 
most affected by the decision. 


DON’T establish decisionmaking proce- 
dures haphazardly or whimsically. 


DON’T allow decision-making processes to 
be established unilaterally, even if this saves 
time. 


DON’T forget that any established proce- 
+ dure must recognize that the principal has 
the ultimate responsibility for decisions at 
the school level. 


DON’T” forget that informed, deliberate 
decision making has rewards that unsystem- 
atic decision making does ndt have. 


DON’T forget the constraints that affect 
the kind of decisions that can be made. 
These constraints include the. following: 


© School code 

© Contracts 

© Budgets 

© Board policies and procedures 


e DO make the decisions on the ‘dates that 


you set during the monitoring. 


DO develop a decision-making process that 
is visible, usable, and understandable to 
those who will be most affected by any 
decisions to modify the age 


. “DO consider the essential issues of estab- 
* lishing e procedures, such as 


the following: 
o Who make decisions? 


© What relative weight will each decision- 
maket’s opinion have? ’ 


© Who will make what type of decision? 
Principal 
Other administrators 
Entire decision-making group 
Grade levels or departments 
Individ (t ers, parents, folun- 
teers, of aides) 


© Will anyone have veto power? . 
© What kind” of decision-making process 
will be used? 
Who will provide input?® 
How will input be provided? 
What will be defined as a problem? 
What will constitute a decision? | 
Will voting, polling, or secret ballgting 
be used? 
How would a decision be revised, 
altered, afhended, or reversed? 


© \What rules about silence and conflict os 
ted? 
will conflict or . diversity be 
handled? 
How will different opinions be- 
-accepted by members? 
How-will conflicts be resolved? 


Don'ts 


@ DON’T assume that everyo, 1 should be 


@ DON’T decide to maké 


» DON'T attempt td make comprehensive -.. 


involved in every decision. 
* a 


changes in the 
_ program solely on thg/ basis of whims, 
politics, or unilateral pefteptions. 


i 
: 


changes, unless you are sure they are 
yyarranted. 


@ DON’T necessarily use the fist. (or the 


easiest, cs ha or safest) way to change 
the program. . 


Do’s 


DO make decisions by‘ using input from 
those with the most expertise in the given 
area. 


DO consider the evaluatiog findings in 
determining the need for and direction of 


_ change. 


DO donsider ‘other factors besides the 
evaluation findings that affect the course 
you ultimately select. These include aspects 
of cost and feasibility (politics, logistics, 
compétence, and motivation). 


DO. choose ‘among the needs for change 
only those that are the most feasible and of 
highest priority. 


DO use the decision-making: process to 
generate alternative ways’ of making 
changes. . 7 


DO select from the alternatives you gener- 
ated using criteria such as the following: 
Creativity . : 
Efficiency 
Effectiveness 
Practicality 
Feasibility 
Economy (time and resources) 


Don'ts 
© DON'T arbitrarily or impulsively select 
“your ¢gourse of action.and thus bypass the 


decisign-making process that you have 
established. 


| 0s 


@ DON’T think that just declaring that a 


Responsibility for making a change should 
be assigned. 


@ DON’T think that the dissemination of 
program changes is excessive communica- 
tion of information overload. = 


& 
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change -will be made will bring it about. ~ 


eo 


Do’s 
@-DO keep track of the modifications you 
make in the program throughout the year, 
and check to see how they occurred. You 
should ask youfself the following questions: ., 
© Did you anticipate the changes in moni- 
toring by asking the right questions 
about your program? 
© Did you make decisions-that considered 
and reflected the evaluation findings? 


© Did you use the established decision- 
smaking process to determine the need 
for change? 
© DO implement the changes that you have 
selected. . 4 
®-DO make your written ‘plan current by 


amending it to reflect the new program 
modifications. 


@ DO engure that all the needed information 

“about program mbdifications is cammuni- 
cated to those who will be affected by 
them. 


¢ 


An Outline for Decision Making 
- Evaluation findings are used in the ongoing planning process to 
make decisions to modify the program, 3 
e following steps in decision making may be gdapted for your 
own situation. You can use them in making decisions to modify your 
program. : ¢ 
A. Establish procedures for quaking decisions “to modify the 
program. 
1. Decide who participates in what kind of decisions. “ 
2. Decide what is required for a detisior¥.-¥ 
‘3. Determine what kind of decision- making’ os will be: 
used. < 
Decide what the rules about silence and sontlior will be. 


. Use the decision-making probess established for modifying the 


program. ' 
1. Involve participants in decisions that require their respective 
experfise. 


2. Determine and prioritize the needs for change from the’ 
evaluation findings. 

3. Generate alternatives, and select ‘Secon among them, con- 

sidering constraints such as feasibility, law, policy, or 

contracts. a 

Implement the program modifications by making the plan 

current and proceeding accordingly. ® 


oot 
@ DONT assume ‘that ongpi 
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= \ | . 
ye \ | 
ov Don'ts 


@ DON’T get in the rut of having one group 
of individuals, who may be energetic and 


and homogeneous, do everything. 


: @ DON'T drop everything| you have been 
"doing to begin the ongoing planning pro- ° 


gS; instead, feed the. i 
tain through various means .into the 
process (such as boar ports, internal 
“Monitor and Reviews,” ang the like). 
- 


© DON’T assume that the 6n y way to have 


involvement is to have on / one person 
make dll the decisions or fo have everyone 
involved in all the decisions, 


| 


‘means. more -paperwork and \documenta- 


tion. It should require less int e Jong run, ~ 


~ if not'Zenerally, because many 
i are actions, thought processed, 


ations. | \ \¢ 


of the steps 


~ 
- 
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Z Do" P 
© DO establish a core group of supporters 
that is representative, internalized pro- 
cess can be developed around this group, 
and participation can be axpanded. 


©@ DO ensure that different types of expertise, 
such as évaluation personnel, are repre- 


| sented. g . 


®@ DO begin or continue using internal “Mahi 
ettor and Reviews” fo provide additional 
information for ongoing planning. 


® DO consider using coniponent committees 
to expand responsibility and to increase 
involvement in the ongoing planning pro- 
cess. To increase communication, members 
of the committed ould be rotated. 


© DO be aware of the benefits of an ongoing 
planning process. They are the. following: 

© Less pressure, rush, and “springtime has- 
sle” to prepare the next year’s school 


e. plan 


Pe -® © Enrichment of what is already happening 


° Team-building and cooperation 

© Responsiveness to program concerns of 

_ Participants 

©!Opportunity ‘to distinguish between 
what is working and what is not and to 
build on what is working 


a ' 


: Ongoing Planning Format: 


This section consists of worksheets on which an ongoing 
monitoring, evaluation, and decision-making process has been 
illustrated.” Thesé worksheets are related closelytto Form 


A-127ES, page 7; which represents a \condensed ° version af . 


the total ongoing planning process. 

Two examples, one for an instructiqnal component ‘and 
one for an instructional support component, are provided on 
the worksheets. By using such worksheets and using the 
‘examples as illustrations but not as models, you. should be 
able to refine your ongoing planning process and to use it 
effectively throughout the year. 

These forms will illustrate ongoing telah ‘and the 


relationships of the epee phases better if the pages are 


_ attached (end to end). 
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Questions, concerns, 
issues, or solution 
procedures 


How closely do the 
prescribed activities 
and matenals for 
mathematics relate to 
the diagnosed needs 
of students? ; 


How effective is the 
present trainis pro-- 
gram for. confining 


pare volunteers? 
. 


* observed in the classroom t 


* EXAMPLE I, 


Diagnosed needs, of students (Individual students are 
determine the frequency of 
diagnose\ and.the variety gf instructional methods, mate- 


rials, and. activities. ). ] 
Teaghers’ descriptions of the Processes they use sd the 


success they have in matching students’ needs to 
activitles . - 


Perceptions of individual students, aides, and parent 
Volunteers concerning the match between prescribed 
activities and materials and the diagnosed needs of 
students 


EXAMPLE IL « 


Volunteer toordinator’s observatign of the degree to 
which.skils addressed in the traiming program are being 
used \ 


Volunteers’ responses to questionnaire evaluating the 
_Process and content of training immediately following 
training and three months following training 


Teachers’ responses to questionnaire regarding changes i in 
the level S volunteer skills 


-November 


Person(s) 
involved 


September Members of 
through 
November matics com- 
mittee, 
teachers 
aides, stu- 
dents, and 
parent vol- 


unteers- 


September 
through 


September * ; 
through 
November 


Parent volun- 
teers and 
teachers 


August 


through 
October _ 


Adgust and * 
November 


October 


the mathe- . 


Person(s) 
responsible 


Administra- 
tor and 
members 
of the 
mathemat- 
ics Com- 
mittee 


Volunteer 
coordinator 


. . 


Person(s) in- 
volved in 
interpreta- / 


Criteria 
for... 
success 


Factors to be considered 
in interpreting results 


Organization of collected 
information 


Analysis of 
information, 


“- 8 ; : EXAMPLE I 
Close matches ot_| Organize collected informa- 
prescribed activi- ] ‘tion on the basis of fre- 
ties and mate- quency of diagnoses;, ° 
‘rials and diag- variety ef instructional * 
nosed heeds of methods; materials and 
students indi- ' activities; teachers’ pro- 
cated in majority] cesses;successes;and , 
of cases perceptions of students, 
aides, and parent 
volunteers. _ 


Quality of instructional 
materials 

Types of students’ needs 
emphasized 

Availability of materials 

Teacheg expertise in using 
diagnosis and prescription 


Compare information by 
grade levels, achieve- 
ment levels, various 
learning needs, and sex. 


Compare parents’ and 
students’ perceptions of 
match to those of { 
teachers and members 

of the mathematics 
committee.. 


Mathematics 
committee 


“ 


EXAMPLE II 
Analyze responses and * 
observation reports by 


process information and 
content information. 


Number of volunteers inteer : 
trained gordinator, 

Number of volunteers 
responding to the 
questionnaire 

* Degree of involvement of 

parents and teachers in 

planning the training 


Some observable 
or indicated 
change in the 
performance 

+  * of volunteers 


Organize collected informa- 
tion on the basis of the 
various volunteer coordi- 
nators’ observations, vol- 
unteers’ questionnaire 
responses to content and 
process (immediate and 
after 3 months), and 

teachers’ responses to 

questionnaire. 


Compare teachers com- 
ments and vplunteers’ 
responses ser 3 months. 

Categorize all information 
collected by grade level. - 


bo 
re 


8I 


W8 11-2 OOE WE90—£0 601-92 


+ mittee and school 


Decision Making 


Persorms) providing 
input and alterna- 
tives for decision 


Anticipated change( s) 
to be made in present 
program ~ 


_Person(s) responsi- 
ble for implement- 
, ing decisions 


a 
Person(s) responsi» 
ble for:decision 


Anticipated changes) to 
be made in next year’s 
program | 


EXAMPLE I 


Administrator, mathe- 
matics committee, 
and staff 


Mathematics com- Greater emphasis will be 
placed on skill develop- 
mefit, concept develop- 
ment, and application 
of knowledge. 

Alternative materials will be |’ 
developed for those skill 
areas in which students 
have problems. ; 


Successful and less success- 
ful processes for mateHjing 
diagnoses and students 

activities will-be identified. 


° 


committee : 


staff* 


\ 


a ae EXAMPLE Il 


Volunteer coordi- 
nator, volunteers, 
and teachers 


Volunteer 
coordinator 


The training of volunteers 
will be modified so that 
their knowledge and skills 
can be more effectively 
applied. 


Volunteer coordinator | Additional training will be 
, given in those skill areas 
needing emphasis,.as iden- 
tified in the evaluation. 

— group sessions Will be 
held. a>. 


*School administrator has final responsibility —but to whom is it delegated? 


Sik os 


